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Abstract 

Three dime nsional surface measurements of large 
objects are required in a variety of industrial processes. 
The nature of these measurements is changing as optical 
instruments are beginning to replace conventional 
contact probes scanned over the objects. A common 
characteristic of the optical surface profilers is the trade 
off between measurement accuracy and field of view. In 
order to measure a large object with high accuracy, 
multiple views are required. An accurate transformation 
between the different views is needed to bring about 
their registration. In this paper, we demonstrate how the 
transformation parameters can be obtained precisely by 
choosing control points which lie in the overlapping 
regions of the images. A good starting point for the 
transformation parameters is obtained by having a 
knowledge of the scanner position.The selection of the 
control points are independent of the object geometry. 
By successively recording multiple views and obtaining 
transformation with respect to a single coordinate 
system, a complete physical model of an object can be 
obtained. Since all data are in the same coordinate 
system, it can thus be used for building automatic 
models for free form surfaces. 

T. Introduction 

The inspection of industrial components for process 
control is of crucial importance for manufacturers. The 
use of optical sensors for this inspection is desirable 
because of the speed and flexibility offered by non- 
contact sensors[lj. However, with this flexibility comes 
a trade -off between accuracy and measurement area. To 
get accuracies comparable to those obtained with 
contact probes, only very small areas can be measured. 
Using common area detectors with 640x480 pixels, the 
width of the measured area can only be up to about 2000 
times larger than the necessary surface height 
resolution. Larger areas can be measured using larger 
area detectors, but the cost of the sensor then rises to 


unacceptable levels. 

When large objects have to be modeled, multiple 
views are needed to get complete surface information. 
Different views with overlapping regions must be 
integrated to form die solid object This involves 
determining the transformation between the different 
views. This process is called the “solving for the 
correspondence problem” and can be accomplished 
by registering the multiple views. There has been 
work done on registration in the computer vision 
comm unity [2, 3, 4]. In most of these cases, the object 
is moved relative to the sensor, without taking 
advantage of the object’s geometry for choosing 
vantage points. It is also important to focus on free form 
surfaces instead of classical surfaces to more accurately 
serve the needs of industry. 

The objective of this work is to provide a starting 
point for the development of a system that would 
ultimately automatically identify overlapping regions 
and perform the registration without requiring 
precision fixtures or precise sensor motion control. 
The system should not be dependent on the object 
geometry and should be able to handle any free form 
surface. 

We approach this problem by assuming that a 
good burial approximate transformation is available. 
This assumption prevents the need to search the entire 
transformation parametric space. This initial 
’transformation can be obtained by tracking the 
camera position or by using machine vision 
techniques to identify surface features common to 
each view. Knowing corresponding points permits the 
use of the alignment method to obtain the ini ti al 
transformation [5]. A good starting point brings the 
views to approximate registration and helps the 
registration algorithm find the global mi nimum . 

Our approach is based on the method developed 


1 




Fig 1. The cross section of a plane containing ball 
bearings is shown in the overlapping region after 
surface A has been transformed with the registration 
transformation. The error in the overlapping region is 
plotted. 

by Chen and Medioni[6]. The registration algorithm 
refines the initial six parameters, three in rotation and 
three in translation, using Newton’s method of 
linearization and iteration to perform least-square 
minimizati on. Arbitrarily selected control points are 
chosen from the overlapping region to estimate the 
correction to rhp. initial transformation. The distance of 
the control points from the reference surface is used as 
the error function. 

To begin our study we used synthetic surface data. 
This data simulate s range images from a laser scanner. 
The data consist of 640x480 element arrays containing 
the simulated object height Z = f(x,y) from the reference 
plane, where x and y are the pixel coordinates. We have 
initially addressed translation and rotation 
transformations, and intend to incorporate scale changes 
in the future. 

IT. Registration 

The registration of multiple view range images 
have applications in 3D object recognition, terrain 
mapping, medical imaging and industrial inspection. 
Many methods that have been used utilize some kind of 
feature correspondence to obtain the transformation 
between views. Such methods require knowledge of 
point to point correspondence between surfaces and are 
very difficult for objects with free form surfaces. 


Idea lly, if the rigid transformation between the 
different views is known, there will be no error between 
the surfaces in the overlapping region. Figure 1. shows 
cross sections through the overlapping regions of two 
surfaces that have been registered. The error plot does 
not have discontinuity which is an indication of good 
registration. If we have two surfaces A and B that 
represent the same object, there exists a rigid 
transformation T between them such that 


V(a e A, 6 € B) | II r« - i II = 0 (1) 

Le., when points a are transformed with a rigid 
transformation T, all transformed as in the overlapping 
region match precisely with the corresponding point b 
of B. T is of die form 

T = 


co sot cosy sinoc sin P cosy— cosot sin y 
cos p sin y sinasinpsiny+ cosacosy 
—sin P sin P cos P 

0 0 


cos a sin P cos y+ sin a sin y r x 
cosasinPsiny- sinacosy i y ^ 

cosacosy t z 

0 1 

where t„ t y and t z are the translation parameters and a, P 
and y are the rotation parameters. These parameters have 
to be evaluated such that equation (1) is satisfied. This 
problem can also be expressed as m i n imiz i n g 

D(A,B) = X II Ta (3) 

where / is the correspondence matching function. 
Solving for T is difficult as it is a highly nonlinear 
function when /is unknown. Searching the parametric 
space will not ensure global minimization. Unless we 
have a definite geometric surface, the function / is 
unknown. The method of evaluation of the registration 
parameters is discussed in the following section. 
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Fig2. The distance d s of the transformed control point 
from the plane passing through the matching point on 
surface B is shown. The normal at the control point is 
h and n, is the normal to the tangent plane. 



Fig 3 shows the method used to determine the inter- 
section point on the reference surface. The shaded 
area represents the search area in the neighborhood 
of the control point on the reference surface. 


A. Selecting a c riterion for registration 

When two surfaces are in registration, the distance 
between the surfaces in the overlapping region must 
ideally be zero. Since the ideal rigid transformation for 
registration is unknown, the registration parameters are 
evaluated such that the square of the distance between 
the two surfaces is a minimum. For a £ A and b £ B 

« = X d t (Ta-b ) (4) 


where d, is the distance of the translated points of 
surface A from B. If N control points form surface A are 
chosen and transformed, the corresponding points on B 
must be identified to estimate the distance error. Points 
b: should be found such that Xll d/T^-b) II is a 
minimum for all r=l.~N. Starting with a good initial 
transformation 1°, an iterative algorithm can be 
formulated. For each iteration k, the previous value of 
7*"* is used to find b* f 

«*= ( 5 ) 


b) = f>imin||7*' 1 « i -fc|| 

The distance d s between the transformed points 
from tangent plane at the estimated matching points on 


B gives the error measures (see Figure 2). The 
distance of the transformed control point from the 
tangent plane is evaluated along the direction of the 
normal of the tangent plane. The tangent plane Sj 
approximates the surface at that point. The distance 
of a point from a plane can be expressed as a linear 
function of its coordinates. The error e k function is 
now defined as 

«*=£<£ ( 2 *a,ty ( 6 ) 

i= 1 

The corresponding points on B must have 
approximately the same z coordinate and the 
normals should have nearly the same slope. Chen 
and Medioni used the surface normal of the 
transformed control points and found the intersect 
era the reference surface by an iterative method. In 
our approach, the intersection point is found by 
searching the coordinate space of the reference 
surface B in the neighborhood of the transformed 
control points. The normals can be compared by 
evaluating the scalar product of the normals of the 
control point and the matching point (see Figure 3) 
and seeing how close they are to one. This method 
avoids the problem of an iterative search. 

For each iteration, the transformation matrix is 
corrected and a better matching point is found. The 
only restriction on this method is that the control 
points should be chosen in the overlapping region. 
The estimation of the correction to the registration 
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parameters is discussed below. 

B. Determining the c orrection to the parameters 

Let the control points chosen from surface A be a cp and 
let a cp be the control points after they have been 
transformed by T. 

V = T '°cp (7) 

Let the equation of the tangent plane at the approximate 
intersection point be 

Ax + By + Cz + D = 0 (8) 


measured error equals the sum of all changes 
in the error resulting from the parameter correction. By 
satisfying all these constraints simultaneously, the error 
can be reduced to zero. The number of control points 
must be greater than the number of parameters to be 
estimated. The more the number of points taken for 
estimation, the estimate will be more accurate. The 
correction A is estimated such that the residue 

min\\Jh-e\\ 

is minim ized. 

This can be reduced to the form of normal equation 
h = ( J T J)~ l J T e (12) 


Each point from a cp ' should ideally lie on tiiis plane. By 
substituting the values of x, y and z of a cp from (7), the 
registration parameters have to be estimated. Instead of 
solving directly for the nonlinear parameters, we can use 
Newtons method to estimate the correction parameters 
as formulated by Lowe[7]. 

Let e be the error measurement between the 
reference image and the translated control points a cp . 
Let * be the correction that has to be made to the 
parameters p. Since we are in approximate registration, 
the assumption of local linearity is valid. Based on this 
assumption, the effect of each parameter correction A,- 
on the error measurement will be A,- multiplied by partial 
derivative of error with respect to that parameter. 

Jh = e (9) 


For iteration, h is evaluated and is subtracted from 
the parameters of the previous iteration. 

p < t+1 > = p (k) -h (13) 

This that the original nonlinear function is 

locally linear over the range of typical errors. 

C. The Registratio n Algorithm 

Take two surfaces A and B and consider B as the 
reference surface. N Control points are chosen from the 
overlapping region of A and are transformed by init ial 
transformation T°. The equation (6) now becomes 

e k = £ d] (J • T k ~ l a cpl sf ) ( W ) 

i= 1 


where: 


where J is the Jacobian matrix 


/ 


v 



( 10 ) 


is the transformation matrix after k\h 

iteration. 

Sj k is the tangent plane on surface B at the b* 


Since the parameters are close to registration, s mal l 
angle approximations can be made. The Jacobian matrix 
is given below 

de- de . 5^ 


5T,' 3,, '3., 


c 


( 11 ) 


is the matching point on B such that it has nearly the 
same z coordinate and • *n s =1. 

is the transformed normal of the control point and 

n s is the normal to the plane at S k . 

7 i cp / «.n J is the scalar product of the control point normal 
and the surface normal at B. 

a ep are the control points from A 
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TTT. Integra tion of multiple vfogS 


d s is the signed distance of from the plane Sf . 

The registration algorithm is as follows: 

1. Choose control points from A such that they lie 
in the overlapping region. Compute the surface normal 
at ca di of these control points and store their position 
and orientation. 

2. For each iteration repeat the following: 

i. For each control point 

u Apply I**' 1 * to *cp' and all its normals. 

u Find the matching points fr*; on surface B by 
comparing the z coordinate and the normals. 

u Find the error between a cp ' and the tangent 
plane at 

u Find Jacobian 

ii. Find the correction parameters h 
iiL i<*> 

The convergence measure is defined as 

6 ,!flziLl se ( S >o) os 

N 

The threshold £ is a direct reflection of the noise level of 
the range image and N is the number of control points 
used. 

T). Control Points 

The registration algorithm uses control points in the 
overlapping region. Ideally, six control points are 
enough to estimate the six parameters, but the system of 
equations is usually over determined for a better 
female of parameters. One of die important features of 
this algorithm is that the choice of control points is not 
dependent on the object geometry. The only restriction 
on their selection is that they should lie in the 
overlapping region because matching points on the 
surfaces are located there. The control points have been 
chosen manually from the smooth regions of the surface 
to be transformed. It is most likely that the reference 
surface will be smooth in the corresponding region. The 
number of control points chosen were between 25 and 
50 depending on the noise in the data and the extent of 
overlap of the surfaces. The registration is better with 
larger number of control points but the time taken for 
the es timati on increases. A study of optimum number of 
control points has not been made as it will depend on the 
complexity of the object geometry. 


hi order to integrate the different views in the 
overlapping region, we averaged the z values at 
particular values of x and y after performing the 
registration. Since the range measurements are of the 
form Z = f(x,y), the surface patches can be tiled 
together to get the complete object description. One 
surface is considered to be the reference surface. By 
registering the other images with respect to the 
reference image, all the images are brought to one 
reference coordinate system. This problem can be 
solved by bringing about registration between 
different views at the same time.In this manner, 
information erf all previous views are used instead of 
just the neighboring views. 

TV. Result s and discussion 

Synthetic 8-bit, 640x480 range images were 
generated to test the registration algorithm. Gaussian 
noise with a standard deviation of 0.945 was added to 
these images to simulate conditions close to real range 
data The algorithm was first tested on two frames of 
data simulating a plane surface studded with 
hemispheres. The two views have three columns of 
hemispheres in the overlapping region. Figure 4 
shows die two images to be registered. We need to 
have prior knowledge of die approximate region of 
over lap as more than one solution for this problem 
exists The actual transformation between the two 
views was 200 pixels in the x direction, 0 in the y 
direction, and 20 units in z. The initial guess was 
given with an offset of up to 21 pixels in the 
neighborhood of the actual transformation. The actual 
transformation between different views were 200 
pixels in the x direction, 0 in the y direction and 20 
units in z. The initial guess were given as 




1 ° = 


1 0 0 2101 
0 10 8 
0 0 1 30 
0 0 0 1 


and die final registration transformation was found to 
be 


T final 


0.999989 - 0.006532 0.004343 199.7546 
0.006473 0.999890 - 0.001103 0.4576 
0.004205 0.001473 0.998999 19.7856 
0.0 0.0 0.0 1.0 _ 
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Rg 4. The two images of the plane with the ball bearings 
is shown.They have three columns of ball bearings in the 
overlapping region. 


Error in z direction 

Rg 5(a). Tlie error image in the region of overlap is 
shown. For a perfect registration, the image will be 
completely dark.(b) The histogram of the residual error 
in the overlapping region. 


The algorithm converged in 4 iterations. It can be seen 
that the algorithm works well for pure translation. 
Rgure 5 shows the error image in the overlapping 
region and a plot of the histogram in the overlapping 
region. 

Another test case was taken which involved 
rotation. Rgure 6(a) shows an object which is 
asymmetrical and (b) shows the same object which has 
been rotated about its axis. The initial guess was given 
as an identity matrix and the registration matrix was 


found to be 


T = 

re i 


0.984697 0.174148 - 0.006636 2.801439 

- 0.174012 0.984589 0.017416 - 2.108876 

0.009567 - 0.015994 0.999826 0.526710 

i 0.0000000 0.000000 0.000000 1.000000 


The error image after registration is shown in figure 
6(c). The histogram of the error is shown in figure 6(d). 
The angle by which the object is rotated comes to about 
9 degrees. 






Fig 6. The object shown in (a) is taken to test the capability of the algorithm for rotation. Tbe object rotated about its own 
2 axis is shown in (b). (c)The error image after registradon.(d) The histogram of the z error in the overlapping region. 


The results with the synthetic data show that the 
algorithm is capable of handling translation and rotation 
transf ormations. The algorithm converges to a good 
registration. This is important because noise is present 
in the data from all actual sensors. The speed of 
convergence depends on the amount of noise and the 
number of control points chosen. 

One of the main requirements for this algorithm is 
the necessity of a good initial transformation. The 


importance of this can be seen from the first example where 
the possibility of more than one solution exists. For 
complicated surfaces, the algorithm can converge to a local 
solution but miss die global solution. This requirement of a 
good initial guess is one of the main drawbacks of the 
algorithm. Ether the relative motion between the object and 
the camera must be monitored, or m a ch i n e vision techniques 
must be used to identify features common to the multiple 
views. Also, the views must have an overlapping region for a 
solution to be found. 
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V. Concludi ng Remarks 


This registration algorithm has been shown to be a 
good starting point for the development of a system to 
construct a free form surface from multiple sensor views 
with minimal constraints. Control points are chosen 
from the overlapping regions of a given surface and the 
registration transformation is found by minimizing their 
distance from a reference surface. One of the main 
advantages of this algorithm is that the choice of control 
points is not dependent on surface features. 

The extension of this algorithm to cover scaling 
transformations will be studied next to handle errors 
caused by focus changes between views. 
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